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Abstract 



Two pilot studies were designed to det^mine the effectiveiess of linguistically controlled, 
highly visual computer-assisted instruction (CAI) for hearing-impaired high school students. 
Study 1, Using Articles^ evaluated a CAI intervention developed in IBM Logo. Our data 
suggested that state-of-the-art CAI, designed specifically for this population, improved the 
students' ability to use articles with count nouns. This study also attempted to discover whether or 
not thwe were differences in the effectiveness of this CAI approach for three specific populations: 

( 1) students whose primary mode of communication was American Sign Language (ASL), 

(2) students who signed exact English (SEE II),and (3; students who communicated orally. 

Study 2, Choosing Advanced Visual Markers (AVMs), was the first investigation in a 
series of instructional interventions to determine the efficacy of using icons (A VMs) in CAI to 
teach English syntax to hearing-impaired high school students. Created using HyperCard on the 
Macintosh, the AVMs were designed to communicate the essence of a syntactical structure to be 
taught, e-g. a native or interrogative pattern. Choosing A VMs evaluated the responses of four 
high school populations to eight different icon designs. In an interactive jwogram the students 
woe asked to rank the alternative AVMs that connoted -negation" ot "interrogation." As in 
Using Articles, this study used three hearing-impaired populations to determine whether or not 
there were differences in the ranking of the icons by students with different, primary modes of 
communication. Preliminary findings suggested that students chose AVMs which were closely 
related to their language background. liiose exposed to ASL selected AVMs which reflected the 
facial expression of a native ASL signer, while those who communicated orally selected the 
universal symbols found in English texts. 



Introduction 



The fbraiidable difficulties that the deaf experience in learning English are reflected in the 
large n;«nbers of functionally illiterate deaf adults. Instructional methods to date have not 
effectively developed the ability of the deaf to read and write English sufficientty. On the average 
deaf adults leach only a fourth grade reading level, and only 1 0% of the best 1 8 year-old deaf 
students read at or above the eighth grade levd. Now, however, as an outgrowth of the seminal 
studies by Noam Chomsky in the late 1960's, the revolution in linguistic thinking has extended to 
sUidies of language and commmunication of the deaf. The pioneering woric of William Stokoe 
(1960, 1975), foUowed by Bellugi (1972, KUmaand Bellugi,1979), Newport (1977), Siple 
( 1978), and Lane and Grojean (1980), firmly established American Sign language (ASL) as a 
"natural" language. Where English is a natural aural-oral language, ASL is a visuai-gestural 
language which evolved to meet the spedRc communication needs of the deaf. Building upon ASL 
conceptualizations, the deaf may be able to acquire the linguistic structures with which they 
ejq)erience difficulty in English but which they acquire with facility m ASL. 

In addition, the availability of new computer technology, which can be used for instruction, 
may remove many of the limitations with which educators and researchers have struggled. While 
hearing children enter school with a fairly complete knowledge of the syntax and lexicon of the 
English langw^e, hearing-impaired students must simultaneously learn the language, and learn 
how to read and write that language. Without ever having heard English, students are expected to 
replicate the lexicon and syntax. Microcomputer-based langui^e programs, however, may offer a 
way to capitalize on the prior langua^rdated, visual knowledge that the hearing-impaired acquire 
in learning sign. Where English uses time and sequencCv the computer courseware designer can 
use the ASL dimensims of position and motion in space to enhance language learning. Even 
though ASL functions in a visual-motor modality and English opecabss in an auditory-vocal 
modality, educational software may offa- hearing-impaired students more rewarding opportunitcs 
for interactive, language expaience than they usually experier^ce. Maximizing the ASL aspects of 
visualization and simultaneity of expression, coraputar-assisted English language pix)grams can use 
color, gR4)hics, and windows to highlight and emphasize instructional points and informational 
feedback. Through CAI designed specifically for the hearing-impaired, a student may be 
motivated to interact with a rich language environment in which the syntax, vocabulary, and 
figurative language are linguistically controlled and incrementally graduated in terms of difficulty 
for hearing-impaired learners; language acquisition and usage can proceed at the learner's own 
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pace. This p^r reports the results of two related studies which were designed to ascertain the 
effectiveness of CAI created specifically for hearing-impaired students. 

Study 1: Using Articles 

Programmed in IBM Logo to run on an IBM-PC with 256K, Using Articles was a 
preliminary investigation of the potential of CAI for effective instructicm of English syntax with 
hearing-impaired students. The computer program was field-tested to answer two basic questicms: 
First, aui state-of-the-art educational software imiwove the ability of hearing-impaired learners to 
use articles with count nouns? Second, will thae be differences in the effectiveness of this CAI 
approach for the three different hearing-impaired populations? 

Sample 

Three hearing-impaired high school populations comprised the study sample for Using 
Articles: 

1 . Ten '•Qral-only** students, ages 14 to 1 8, were in a day program for the 
communicatively handicapped Oral-only indicates that the students* 
first language is English and that they conununicate in an aural-oial 
mode. 

2. Ten "Total Communication" students, ages 14 to 19, used SEE II and 
speech in a day program. SEE II, Signing Exact English, is a type of 
Manually-Coded English. 

3 . Nine "Total Communication" students, ages 14 to 16, were enrolled in a 
residential program, which used ASL or PSE (Pidgin-Signed-Biglish) 
as their language of choice for pars<»ial communicati(»i. 

Methodology 

The program. Using Articles, was limited to three lessons which could be completed by an 
average subject in approximately one hour. The lessons consisted of exercises and paragraphs 
which focused on: 

1. Choosing the article 3 or an 

2 . Using a, gQ, and the null article (0) with singular and plural count 
nouns; for example, "balls" are "countable" vvhUe "air" is not The null 
article indicates the lack of an article preceding plural count nouns; for 
example, he eats an apple every day, but he never eats 0 pears. 



3 . Using a, aa, and the null article (0) in first and second mention. 
The first time a noun is mrationed, an indefinite article is used, but the 
second menti(Mi requires a definite, article; for example, I found a key. Is 
it the key to your car? 

At each of die three test sites, the goals of the program were explained and the students' 
support was elicited After penciUand-p^per pre-tests were administered to each group, students 
proceeded through Using Articles individually; ttie learner controlled the pace. Within 24 hours of 
using the program, a post-test was administered. The format, level, and administration of the pre- 
and post-tests comparable. The syntactical structures of the sentences in the pre- and post- 
tests were identical, but the nouns and verbs were different 



Results 



Figure 1 reports the pre- and post-test scores of each gioup. Repeated Measures Analysis 
of Variance (ANOVA) showed that the three groups demonstrated improvemwit in ail three topics. 
All pre- post-test differences were significant (p < .05). 

1» A and An — Although significant, the slight improvement between 0 
and 1 errors indicates a likely ceiling effect The three groups seem not 
to have needed instruciion on this topk. 

2. Singular and Plural ~ As in tha ra^ of "A anH An/' th^ Oral^ly 

group seems not to have needed this lesson. As demonstrated by the 
number of mistakes on the pre-test, the SEE II and ASL groups needed 
and benefited fipom this lesson. Eight of the nine ASL students, and the 
ten SEE II students demonstrated improvement Considering tfie 
brr^vity of the exposure — only one hour of intervention — it is 
surprising to see such a significant degree of improvement within these 
two samples. 

3. First and Se cond Mention The number nf ^rmrg in the pm-test 
indicates that all three groups needed and benefited from instruction on 
this topic. Usually, hearing-impaired students with the most language 
make the most progress; however, in this difficult lesson the rate of 
improvement appears equally high for all three groups. The ASL and 
S EE n groups i mproved at a rate equal to their performance in the 
"^Singular and Plural** lesscHi. However, the graph displaying this result 
masks substantial variance in the individual performances: more than 
half the students displayed strong and steady progress, one quarter 
(mostly from the Qral-<Mily group) did not seem to need the instruction, 
and the other quarter did not benefit from the instruction. The mixed 
results from Uiis most difficult lesson, Tirst and Second Mention," may 
be attributed in part to a need for more practice. Because there was a 
fixed amount of jMactice (one hour) across all three topics, one can 
i^ulatc that additional practice in the more difficult areas would lead to 
an increase in learning. 
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Lessoa 1: A and N 
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10 
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Lesson 2: Singular and Plural 
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Lesson 3: Fixst and Second Mention 
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Figure 1. Mean Scores on Pre and Post-tests and Decrease in 
Mean Number of Enors for Three CAI Lessons 
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Cwisidering the usual slow rate of improvement of syntactic s juctures among the hearing- 
impaired (Quigley, J 978), the encouraging findings from this carefiilly designed CAI intervention 
of shat duration motivated us to study other aspects of ASL which could be transferred to an 
advanced computer technology. Our investigation led us to HyperCanl on the Macintosh, which 
combines the ease of use associated with a graphic interface with the power of an object-oriented 
programming language. Critical components of our instructitml approach, graphics and 
animation, are handled with fiacility and speed in HyperCard; presentation of the instructional 
design is delievered with appropriate speed on the Macintosh. 



Study 2: Choosing AVMs 

Programmed in HyperCard to run on a Macintosh, Choosing AVMs (Advanced Visual 
Markers) was 'ihe first investigation in a soies of instructional intervaitiras to determine the 
efficacy of using icons (AVMs) to teach English syntax to hearing-impaired students. (See the 
Icon Kqt in Figures 2 and 3.) In English the grammatical element tiiat distinguishes a declarative 
sentence from an interrogtitive or negative one is syntax, i.e. word order, howwer, in ASL the 
grammatical signals that in.dicate sentence type are the signer's facial, eye and head behaviors. 
These highly visual, non-manual behaviors are produced concurrentiy wiUi all the ASL signs for 
the concepts and words in the sentraice (Baker, 1980). 

To capitalize on ASL's visualization and simultaneity of expression in teaching English, we 
created AVMs (icons).to provide learners with a visual clue, or bridge. The AVMs were designed 
to denote the English language requirement for a tiansfOTmation, a change in word order, when 
writing a negative sentence or question in English. 

The first majw st^ in developing instructional materials built around AVMs was to decide 
upon the optimal style of icon. We designed eight AV^s (icons); four for negation and four for 
intOTOgation. TTie AVMs ranged from abstract symbols to line drawings to the digitized 
photographs of a native ASL signer's facial expression when asking a yes-no question or 
expressing negation. Figures 2 and 3 display the negative and interrogative AVMs Uiat wctb used 
in this pilot study. 
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Figure 2. Percentage Ranking of Interrogative Icons by School Site 
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Figure 3. Percentage Ranking of Negative Icons by School Site 
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Choosing AVMs was designed to evaluate the responses of four high school populations to 
eight different AVM designs. In an interactive program the students were asked to rank the 
alternative AVMs that connoted "negation" or "interrogation." The study sought to determine the 
answers to two basic questions. First, would one AVM emerge as the dominant choice across all 
student populations? Second, would students choose AVMs which were closely related to their 
language background? For ejcample, would students with exposure to ASL tend to choose Icons 4 
and 3 for interrogation and 8 and 7 for n^ation because those AVMs were derived from the 
digitized image of a native ASL signer's facial expressions? In contrast, would the more aural- 
oral, mainstreamed students select the universal, textbook symbols in Icons 1 and 2 for 
interrogation, and Icons 5 and 6 for nation? 



Sample 

The sample for Choosing AVMs was drawn from four secondary school, hearing- 
impaired populations to determine whether or not there were differences in the ranking of the icons 
by students with different, primary modes of communication: 

1 . Twelve "Oral-only" students, ages 14 to 1 8, were in a day program for 
the communicatively handicapped.at Gunn High. 

2 . Twenty-five "Total Communication" students, ages 14 to 19, used SEE 
II and PSE with speech in a day program at Leigh High. 

3. Thirteen "Total Communication" students, ages 14 to 19, used SEE II, 
PSE, some ASL, and speech in a day program at Capuchino High. 

4. Twenty-one "Total Communication" students, ages 14 to 19, used PSE 
and ASF with some speech in a residential program at the California 
School for the Deaf in Fremont (CSDF). 



Methodology 

To collea data about the AVMs (icons), we developed Choosing A VMs an interactive 
program in which 71 hearing-impaired students in four school settings assisted in designing their 
own instructional material by indicating their preferences fw the AVMs currently under 
consideration. At. each of the four test sites, the goals of the program were explained and the 
students' support was elicited. Each student proceeded through Choosing AVMs individually; the 
student controlled the pace. Following several interactive screens that explained the Macintosh 
interface, Choosing AVMs displayed eight sentences with corresponding pictures. Each sentence- 



picture combinaticn w-as displayed vA\h each of the four appropriate A VMs, for negation or 
intOTOgation. The presentation order of the AVMs was randomized but did not include the 
possibility of the AVMs being presented as a continuum from abstract to concrete or vice-versa. 
After each sentence-picture-A VM display, the students were asked to rank the four AVMs 
according to the following criterion: which AVM best captured the essence of n^ation — or 
interrogation — for that sentence-picture? Data were collected on the presentation oider and the 
ranked responses. 

Results 

Figures 2 and 3 show the students' percentage rankings of their choices of interrogative 
and negative AVMs respectively. Let us first describe the students' first choices by school, and 
then turn to differences across all choices by school. 

The solid black bars on the histograms in Figure 2 show a consistait preference for Icon 1 
— the most abstract, inteirogative AVM for all schools except CSDF. A plausible explanation 
for the difference in preference between students at CSDF and the other three sites could be that the 
students at CSDF have been exposed to and used sign language as a primary mode of 
communication. Because fecial expressiais tend to be incorpOTated as key syntactical elements in 
sign language, the AVMs derived from ASL facial expressions would be more femiliar to these 
learners. 

In contrast, because students at the otiier three sites have been mainstreamed for a number 
of years, their language experiences tend to resemble those of hearing populations. As a result, 
their AVM preferences would be mwe closely related to the abstract punctuaticHi symbols more 
familiar to learners sharing conventional, English language instruction. 

Looking beyond first preferences, of particular note is the pattern of responses displayed in 
Figure 2. While Gunn students showed a progressively decr^ising interest in icons along the 
abstract to concrete continuum, with only 23% choosing interrogative Icons 3 and 4, the CSDF 
students displayed a strong interest in the more concrete ASHike respresentations; 65% selected 
Icons 3 and 4. Again, this preference is likely a reflection of the CSDF students' visual mode of 
learning and exposure to ASL As a result, their preferences for Icons 3 and 4, the line drawing 
and the digitized photographic image of a native ASL signer using the facial expression for a yes- 
no quesion, are quite undmtandable. In contrast, die Gunn students' primary mode of 
communication is aural-oral English, wiUi little or no exposure to ASL. 
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The responses displayed in Figure 3 show a pattern similar to Figure 2. Only 29% of the 
Gunn students expressed an interest in the ASL A VMs, Icons 7 and 8. In contrast, 58% of the 
CSDF students selected Icons 7 and 8. To deaf students, like those at CSDF who heavily rely on 
visualization, fecial, eye and heid bdiavior play a dominant role in their communication. Yet over 
all schools, the pattern of preferred icons was not as consistait for negative sentences as it was for 
interrogative, Capuchino students, like those at CSD, showed a preference for an AVM other than 
the most abstrac*. Further analysis of this data may reveal more about the underlying causes for 
these response patterns. 

Educational Importance of Studies 1 and 2 

The data from these preliminary studies are encouraging. The results of Using Articles 
suggest that state-of- the-art, computer-assisted language instruction may be of benefit for all three 
populations studied. Although secondary school hearing-impaired jjtudents may not need direct 
instruction or inteny ve review of "A and An," most are likdy to require work in the areas of 
"Singular and Plural" and "First and Second Mention." Where needs are demonstrated, innovative 
CAI, even of a relatively short duration, has powerful effects. The fact that pre-post gains were 
similar for all three groups suggests that such programs may enable aU hearing-impaired students 
to improve their English regardless of the degree of their hearing loss. 

Students participating in Choosing AVMs responded positively to the ORwrtunity to 
participate in the design of their own instructional material. Although the AVM preferences among 
the four school sites were significantly different, the study revealed interesting differences withm 
each school population. Rather than assign a dominant AVM to all students at a given school site, 
the program con be designed to allow individual students to select whichever AVM best reflects 
their iniividual prcfensnces. In fact, students could select different AVMs as they jroceeded 
through the instructional series. 

Even though these sttidies indicate the positive effect of linguistically controlled, highly 
visual CAI for hearing-impaired stiidents, the studies - we a number of areas to investigate. The 
next step in our research will be the development of Using AVMs. In order io Unk the English 
syntactical structure to a visual ASL constiiict, we will pair £he AVM witii the English structure; 
computer animation will be used to maximize the tiansfer of relevant knowledge. Multiple 
opportunities for initiating interactive practice will be provided. As the learner becomes proficient, 
tiie AVM wiU fede, enabling the hearing-impaired learner to become familiar with typical 



presentations of English syntax. If a student's performance begins to decline, the icon will 
reappear. 

Using AVMs will incorporate a series of studies of two syntactical structures: n^ation and 
interrogation. Each study will be a contolled expaiment involving the formation of treatment and 
conrol groups with random assignment of subjects to groups. The experiments will all be tests of 
the null hypothesis. We will determine whether we can reject the proposition that the use of AVMs 
has no effect on the amount of syntactical knowledge learned finom a CAI presentation that cannot 
be accounted for by chance. In each study treatment and control subjects will receive almost 
idoitical treatments, the single difference being that the treatmait subjects will have AVMs 
embedded in their CAI presentations. 
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